DOCUMENT RESUME 



ED 475 762 



TM 034 883 



AUTHOR 

TITLE 
PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Knezek, Gerald; Christensen, Rhonda; Fluke, Ricky 

Testing a Will, Skill, Tool Model of Technology Integration. 

2003-04-00 

12p.; Paper presented at the Annual Meeting of the American 
Educational Research Association (Chicago, IL, April 21-25, 
2003). 

Reports - Research (143) -- Speeches/Meeting Papers (150) 
EDRS Price MFOl/PCOl Plus Postage. 

Academic Achievement; ^Educational Technology; Elementary 
Secondary Education; ^Models; Public Schools; *Skill 
Development ; * Technology Integration 
Texas 



ABSTRACT 

The Will, Skill, Tool (WST) model of technology integration 
(G. Knezek, R. Christensen, R. Hancock, and A. Shoho, 2000) postulates that 
enhancing an educator's will, skill, and access to technology tools leads to 
higher stages of classroom technology integration, which in turn leads to 
greater student achievement. In this paper, regression analyses and 
structural equation modeling techniques are presented representing a wide 
variety of data sets from Texas public schools in order to test combinations 
of observed variables that optimize the WST model fit statistics. Data sets 
were: (1) survey responses of 39 teachers in a metropolitan area; (2) data 

from 100 public school districts about funding and student achievement; (3) 
data about integration of technology from 12 teachers and their 170 first- 
and second-grade students; (4) technology integration data from the same 12 
teachers; and (5) data from 1,267 K-12 teachers about classroom technology 
integration. The analyzed of these data sets support the use of the WST 
model. Implications for educational evaluation and policy are discussed. 
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Testing a Will, Skill, Tool Model of Technology Integration* 



by 

Gerald Knezek, Univ. of North Texas, gknezek@tenet.edu 
Rhonda Christensen, Univ. of North Texas, rhondac@tenet.edu 
Ricky Fluke, Univ. of North Texas, adllO@earthlink.net 

Abstract 

The Will, Skill, Tool (WST) model of technology integration (Knezek, Christensen, Hancock & 
Shoho, 2000) postulates that enhancing an educator's will, skill, and access to technology tools 
leads to higher stages of classroom technology integration, which in turn leads to greater student 
achievement. In this paper, regression analyses and structural equation modeling techniques are 
presented representing a wide variety of data sets from Texas public schools in order to test 
combinations of observed variables that optimize the WST model fit statistics. Implications for 
educational evaluation and policy are discussed. 
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Conceptual Rationale 

The school learning environment. 

Learning in school has been recognized as a complex issue for many decades. For example, in 
1975, Klausmeier and Goodwin identified 37 variables in nine major categories that affect the 
school learning environment. Among these were items such as instructional goals and 
objectives, student intellectual ability, teacher interest and knowledge of subject matter, home- 
school-conununity relations, physical space and equipment characteristics, and availability of 
print and audiovisual instructional materials (Klausmeier & Goodwin, 1975, p. 13). 



In 1999, the U.S. Department of Education established a $75 million funding initiative aimed at 
producing 2,000,000 new technology-using educators capable of integrating information 
technologies into daily classroom practice (Preparing Tomorrow's Teachers to Use Technology 
Program, 1999). Calls for gauging the impact of this infusion of technology into the nation’s 
classroom also accompanied the funding, and, in 1999, the Milken Exchange on Education 
Technology identified seven major categories of variables intended to “ . . . help policymakers 
answer the question, ‘What is the return on the public’s investment in K-12 learning 
technology? ’’’(Coughlin & Lemke, 1999, p.38). Many of these are similar to the categories of 
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variables affecting the school learning environment that were identified by Klausmeier and 
Goodwin twenty- four years earlier. 

The search for positive impact 

Much recent research in educational technology has focused on impact.. Researchers have been 
generally successful in demonstrating the positive impact of technology infusion with 
appropriate teacher training on teacher attitudes toward information technology, student 
attitudes toward information technology, and other learning-related student dispositions such as 
motivation (Woodrow, 1992; ColHs, et al., 1996; Christensen, 1997). Instraments have been 
developed by several researchers to aid in this process (Christensen & Knezek, 1998; Knezek & 
Christensen, 1998; Ropp, 1999; Christensen & Knezek, 1999; Knezek & Christensen, 2000). 
Researchers have been less successful in identifying positive impacts of technology infusion on 
student achievement, although some studies have met with success. For example, a review of 
130 studies by Bailo and Sivin-Kachla (1995) concluded that using technology to support 
instraction improves student outcomes in language arts, math, social studies, and science. An 
evaluation of the West Virginia Basic Skills/Computer Education program concluded that “... 
the effective use of learning technology has led directly to significant gains in math, reading and 
language arts skills in West Virginia” (Mann, Shakeshaft, Becker & Kottkamp, 1999). More 
conunonly, however, findings have been mixed. ' 

Method 

The approach advocated in this paper is multivariate in nature, based on the development and 
testing of a formalized model for integrating technology into the classroom. The Will, Skill, 
Tool model of Technology Integration targets the influence of the teacher on the classroom 
portion of the school, and further restricts the area of interest in the classroom to technology 
integration. The model also incorporates the influence of technology integration on student 
achievement. 

Figure 1 contains several indicators of teacher attributes in the areas of will, skill, and tool. Two 
indicators of classroom integration, and three indicators of student achievement are listed as 
well. These indicators are conceptually associated with constracts using conventions that have 
become coimnon in structured equation modeling (Schumacker, 1996). 
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Figure 1. Will, Skill, Tool Model of the Impact of technology integration on achievement. 

Study 1: 1999 Teacher Data 



The 1999 preliminary test of the model used survey responses gathered during the fall of 1999 
from teachers in a public high school in the Dallas Ft. Worth Metroplex area of northern Texas. 
Thirty-nine teachers completed the following battery of instruments: 

• The Teachers' Attitudes Toward Computers Questiormaire (TAC 5.1) (Christensen & Knezek, 
1998) measures seven indices regarding teachers' attitudes. 

• Teachers' Attitudes Toward Information Technology Questionnaire (TAT) (Knezek & 
Christensen, 1998) is a semantic differential instrument that measures attitudes toward new 
information technologies . 

• The Technology Proficiency Self-Assessment (TPSA) (Ropp, 1999) measures technology 
skills in four areas identified by the International Society of Technology in Education as key for 
their National Educational Technology Standards (NETS). 

Two self-reported outcome measures were also included in the battery cf instruments. One was 
the Level of Use Questionnaire (CBAM_LOU) based on the Concems-Based Adoption 
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Model/Level of Use scale for diffusion of innovation (Hall, Loucks, Rutherford, & Newlove, 
1975), and the other was the Stages of Adoption of technology in education scale (STAGES) 
developed by Christensen (1997), based on the earlier work of Russell (1995). 

Three demographic variables indicating level of access to technology tools were included in the 
set of 1999 data available to researchers, as well. These were teachers’ current hours per week 
using information technology in the classroom (CHOURS), whether or not teachers had access 
to a computer at home (HOME_PC or HOMECOM), and whether or not teachers had access to 
the World Wide Web at home (HOMEWWW). 

Results of 1999 Teacher Data Analysis 

Ideally structured equation modeling (Schumacker, 1996; Joreskog, & Sorbom, 1998) would 
have been employed to determine how well the model fit the data, but its use was not attempted 
for the 1999 data set due to the small data set and large number of degrees of freedom in the 
model (R.E. Schumacker, personal communication, December 7, 1999). Therefore, regression 
analysis was chosen to explore how well will, skill, and technology tools could place an 
individual in the proper stage of adoption. 

Impact of Will. Skill, and Tools on Technology Integration 

Impact of Will. A regression analysis verified that approximately 40% of the variance in stage 
of adoption is attributable to “will” measures for these teachers. The R-squared for stage of 
adoption predicted from the TAT attitude scales (EmaUT, WWWT, MMT, ProdT, ProdCl) was 
.39 (Knezek, Christensen, Hancock & Shoho, 2000). 

Impact of Will and Skill Combined . For the 1999 set of thirty-seven secondary teachers, adding 
skill measures to the equation increased the predictability of stages of adoption of technology 
from roughly 40% to 70%. The R-squared for Stages predicted from TAT attitude measures and 
TPSA skill measures is .69 (Knezek, Christensen, Hancock & Shoho, 2000). 

i 

Combined Impact of Will. Skill and Access to Technology Tools. Adding measures of access to 
technology tools for teachers increased the predictability of stages of adoption from 70% to 84% 
for this set of data. The R-squared for TAT attitudes, TPSA skills, and the three tool variables of 



current hours per week using technology in the classroom (CHOURS), access to a computer at 
home (HOMECOM) and access to the WWW at home (HOMEWWW), was .84. 

Study 2: 1999 Funding and Student Achievement Data 

A second data set was compiled to further test the model introduced in 1999. This analysis 
explored the relationship between available variables representing both ends of the technology 
infusion-to-student-achievement continuum. The Texas Assessment of Academic Skills (TAAS) 
test pass rate, and district wide average SAT score, were used as dependent variables. District- 
wide hardware expenditures, and district-wide expenditures on software and technology 
supplies, are the two independent variables used to indicate extent of support for information 
technology facilities. The data were gathered by randomly selecting 100 of approximately 1046 
public school districts in the state and recording their scores from the Texas Academic 
Excellence Indicator System (AEIS). 

SAT Scores. Hardware and software expenditures together accounted for approximately 12% of 
the variance in SAT scores across the sample of school districts (R squared = .115) (Knezek, 
Christensen, Hancock & Shoho, 2000). This relationship is significant at the .05 level (f = 3.25, 
2x50df,p = .0472). 

TAAS Pass rate. Hardware and software expenditures together account for approximately 1 1 % 
of the variance in TAAS pass rates across school districts (R squared = .11). This relationship is 
also significant at the .05 level (f = 3.70 , 2 x 58 df, p = .0307). 

Limitations of 1999 Analyses i 

Only thirty-nine teachers from one school are represented in the data for the portion of the 
model dealing with stage of adoption predicted as a function of will, skill, and access to 
technology tools. A much larger sample was needed to test aU components of this portion of the 
model in a structured equation modeling paradigm. In addition, just one indicator of technology 
integration (stages) had been utilized, but in structured equation modeling, multiple indicators 
for each constmct are preferred. Similarly, only 100 school districts were included in the current 
analysis of the relationship of technology investment to two measures of student achievement. 
The research team concluded that a comprehensive test of the portion of the model dealing with 
the impact of classroom technology integration on student achievement probably required 
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lowering the unit of analysis to the classroom level. These modifications in research design 
were incorporated in smdies 3 and 4 described in the following sections. 

Smdy 3: 2000-2001 Technology and Reading Achievement Data 

) 

Data were gathered from twelve first- and second-grade teachers from a suburban north Texas 
public school district and their 170 smdents during the fall of 2000 and again during the spring 
of 2001. Teachers provided Stages of Adoption and CBAM Level of Use ratings. Woodcock 
Reading Mastery Tests-Revised (WRMT-R; Woodcock, 1998) were administered to smdents as 
pretest and post test measures, using the subtests Word Identification, Word Attack, and Passage 
Comprehension. Stages of Adoption and CBAM Level of Use were used as indicators of the 
latent variable “classroom integration” while pre-post reading difference scores were used as 
indicators of “smdent achievement”. As shown in Figure 2, the resulting path coefficient from 
integration to achievement, in an SEM model with acceptable fit statistics, was .35. 




W cock7.jpg 1 5 March 2002 

Figure 2. Strucmral equation model for Woodcock Reading Mastery Tests score gains as a 
function of level of classroom technology integration. 

Smdy 4: 2000-2001 Technology Integration Data from Teachers 



Will (attitude) and Skill (technology proficiency) and Tool (access to technology tools) data 
from the twelve teachers described in Study 3 were submitted to an SEM analysis to determine 
if classroom integration could be well predicted by will, skill, and tool indicators. As shown in 
Figure 3, Skill appeared to be a strong predictor of classroom integration for these teachers, 
while Will was a weaker but acceptable predictor. Incorporation of the Tool latent variable did 
not result in an acceptable fit statistic for this group of teachers. 




Wc(Kk3,jpg 15 March 2002 



Figure 3. Structural equation model for classroom technology integration as a function of 
teacher will and skill. 



Study 5: 2002 Will, Skill, Tool and Technology Integration Data from Teachers 

Data were gathered from 1267 K-12 teachers representing aU 37 schools in a suburban public 
school district of Texas during April of 2002. Preliminary analysis of this data indicates that 
Will, Skill and Tool latent variables form an acceptable collection of predictors of classroom 
technology integration (as measured by Stages of Adoption and CBAM Level of Use). 
Additional research is underway to gather a comprehensive set of data that will enable testing 
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both the teacher (technology integration) side of the model and the student (learner outcome) 
side of the model from teacher and student data gathered within a set of schools in the same 
timeframe. 



Summary/Conclusions 

While the effectiveness of the teaching and learning process has long been recognized as a 
complicated issue, technology adds another dimension to the assessment of elements that are 
contributing factors to a student's quality of education. A new model for integrating technology 
into the classroom is presented along with analysis showing that technology investment can 
contribute to student achievement. The model includes three key elements for successful 
integration of technology: Will (attitude) of the teacher, Skill (technology competency), and 
Technology Tools (access to technology tools). The model also postulates that technology 
integration in the classroom contributes to higher student achievement. Preliminary analysis of 
this model has yielded promising results. 

The current emphasis on accountability for technology expenditures in the USA has spawned a 
quest for evidence of positive impact on student achievement, especially at the K-12 level. Since 
the primary environment in which learners and teachers interact with information technology is 
evolving to be the classroom, a formalized model of technology integration facilitates 
systematic testing of theorized relationships. One important aspect to be tested is the 
predictability of classroom technology integration based on teacher measures of will, skill, and 
access to technology tools. Another important aspect is the predictability of student achievement 
based on level of technology integration. Findings to date indicate this tv'o-step model is worthy 
of further exploration. 
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